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ABSTRACT 

The purpose of the evaluation is to inform the 
Leadership Training Institute (LTI) staff how well they perforaed 
their responsibilities during 1972-73, The procedures for gathering 
information on which the evaluation is based include; (1) a personal 
interview with each LTI staff B^eaber; (2) personal interview with 
appropriate Bureau of Libraries and Learning Eespurces, United States 
Office of Education (BLLS/USOE) staff; (3) telephone interviews with 
directors of 1972-73 institutes; and (4) a review of products 
produced. Because of the unavailability of participants' naaies and 
addresses, in addition to lack of time and money, only four iaeaUers 
of the LTI staff, two representatives of the BLLR/USOE and 10 
directors were interviewed during July, August, and Septeaber 1973. 
All available products were reviewed during August and September 
1973. The conclusions and recommendations include: (1) roost of the 
1972-73 goals for LTI were achieved to a fairly high degree; and (2) 
the establishffiont of more realistic, less global goals is 
recoBoended. Detailed conclusions are listed, along with 
recoffliBendacions, in addition to appendixes. (kCM) 
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CONTEXT OF THE EVALUATION 

On June 1, 1973 the evaluator was asked to serve as an '^outside" 
evaluator to assess the effectiveness of the 1972-73 Leadership Training 
Institute {LTD which operated under a contract from the Bureau of 
Libraries and Learning Resources (BLLR) , United States Office of Education 
to Florida State University. Since the request came at the end of the 
program, a pp^\^ hoc or summative evaluation v;as mandated. The procedure 
involves: (1) discrepency analysis (the degree to which original objec- 
tives of the LTI were achieved) ; and (2) impact analysis; (the immediate 
effect of the LTI on its primary audience) » 

The primary purpose of the evaluation is to inform the staff of the 
LTI wl^at has worked (and how v:ell) and what hasn't worked. It should 
also servo as anobjoctive report to the training staff of the Bureau of 
Libraries and Lcarniitg Resources. 

The procedures for gathez'ing inforriation v:pon which the evaluation 
is based include: 

1* Personal interviews v;ith each LTI staff member ; 

2. Personal interviews with appropriate BLLR/USOE staff; 

3. Telephone intorviev;s v:ith directors of 1972-73 institutes; and 

4. Review of products produced. 

Content for each series of anterviev;s was reviev.ed by the LTI staff 
to be sure that comprehensive interviews w^ould be conducted and appropriate 
infornation elicited. 
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The ori<jinal plan Ccillod for intorviews with a sainplincj of institute 
participants to dotorniue tlio viltiu^ate effects of the institutes on the 
dasiicjnatcd audiences. The unavMilability of participant's names and 
addressee and the lack of time and rroney to carry the evaluation to this 
audience prevented this stop from being carried out, V/hile these data 
nicjht have been useful, the primary audience for LTI activities is the 
directors and therefore the emphasis v/as placed on this group. 

Four members of the LTI staff, two representatives of BLLK/USOE and 
ten directors v;ere interviewed during July; August and September, 1973 to 
gather data upon which evaluations could be made. All available products 
were reviewed during August and September, 1973. 

The evaluator met with the entire LTI staff on June 26, 1973 to 
reviev; the recomjr.ended evaluation process. The purpose of this meeting 
was to determine the appropriateness of tire issues being explored, the 
utility of the proposed qvicstions, and the acceptability of the proposed 
procediire. As a result of this meeting, the procedures v;ore modified and 
the proposed interview form.ats wore developed. One major criterion which 
permeated the m.eeting was the degree to which evaluation needs of the staff 
could be met through the proposed procedure. 

The jud<^jirents in this report are the sole responsibility of the eval- 
uator. Once t)v: data v/orc gathered, the analysis and interpretations w^ere 
made as objectively as possible. Errors in judgment could bo c^U3sed by 
(1) a sm.all population; (2) inade^^iate data collection procedures; (3) 
insufficient c\ita; or (4) lack of the evalucttor^s j^inceptioii . 
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DISCREPE^^CY EVALUATIOM 



The goals of the LTI are used as the base refetGnco point for the 
evaluation. Goal statements are somewhat broader than statein'^nts of 
ohjectivosr buc the goals stated in the original proposal are suffici-- 
cntly specific to perii^it assessment,^ 

^ ' To ach ieve i mproved leadership training sk ills and conpe- 
tcncieG f or federally funded ins titut e di re c tors and their 
key staffs ; and for o ther per sons designated by the As sociate 
Co rr.m i s s i on e r ^ BLET (BLLR) . 

Evaluat ion . Leadership training skills were apparently 
defined as evaluation competencies and coiTununications 
skills. To achieve this goal, three workshops v;ero held 
and two products were developed: (1) an evaluation manual 
and (2) a communications package. According to the January- 
February, 1973 Report no 3LLR, in tv/o of the three workshops, 
^'almost 99% of the participants felt that the planning/ 
evaluation concepts presented v;ill be helpful in current and 
future training activities." 80h of the participants in the 
third v;ork3hop indicated the utility of the concepts. How- 
over, when thi'i directors were contacted, GC^ indicated that 
they had not used the evaluation concei>ts. 20% indicated 
that they v;ere not able to use tlie evaluation ideas because 



The original pro2^osal stated a series of ^'activities'^ which were 
derived from, the goals. Activities are interpreted as means for 
attaining goals and therefore are not judged specifically. 
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they came too late in the procjravn and 20% indicated that they 
had used some ol' tho evaluation principles. There is clearly 
a dir,crepency botwoeri the perceived usefulness of evaluat:-On 
principles at the; end of the v/otkshop and the actual use after 
that tine. 

Perhaps the terrr.s, "skills** and "competet^cies" are too strong. 
There is no evidence of any director having gained new or 
improved leadership training skilJ s or competencies in evalu- 
ation (or any other area) as a direct result of LTI's efforts. 
If the goal read^ "to create an awareness of new leadership 
training procedures/' or "to provide guidelines for improved 
program operations" it would be easier to report successful 
attainment of the goal, but "skills" and "competencies" - no. 

In another area of leadership training/ LTI provided the 
conditions, through the meetings of directors and other invited 
guests, to gain skills in management of institutes. The oppor- 
tunities given during these laeetings to share ideas and proce- 
dures v;ero i^erceived by most directors as helpful. Again, 
"skills" and "competencies" probably v/ere not gained but nov; 
id-:ias and procedures for institute managcir.ent v;oro exchanged 
and ev^entually used in several institutes. 

Api'Arently sorr:- px"ogross tovrard this goal was Luide in the area 
of coinmunicotion skills. Severe"^! directors rci'ori-cd tliat, as 
a result ot I-TI efforts, they wore able "to work more closely 
v;ith students" and were "more sensitive to student comments." 
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It is difficult to assign cause and effect relationship here. 
Some of the directors were exposed to preliminary versions of 
the corniaunicat ion fjkills packaqe which may have provided new 
insights which were ultimately used^ IJowover, strong positive 
statoTTvents regarding the site visits made by Bene Durant might 
lead one to believe that her attention to student concerns, 
which was pointed out by several directors, might have led to 
the improvetr.ent of communications. It seems that Ms. Durant 
served as an ombudsperson or medium through which infomation 
' and feelings were passed from student to director and director 
to student. Causal factors may not be important here. LTI 
had a role to play in improving communication and attending 
to student concerns* Whether "improved leadership training 
skills and comxj^toncies" in communications wore gained by the 
directors and staff members is another question, 

B . To inventory r asses s^ and estab lis h priorit ies for c riti cal 
tra inin g needs in t he libra ry and media professions > 

Eva J.uation . There is no evidence that any list or documenta- 
tion of training needs exists nor that any systematic procedure 
was followed "to inventory, assess, and establish priorities 
for critical training needs...," The need for training in 
evaluation was poicoived by directors to be determined by LTI 
or BLLR. Act'ially there v;as some indication from the 1Q71-72 
diroctoris that evaluation assif^tance v;an needed. This fact was 
picked up by the 197 2-7 3 staff when the grant v;r.s shifted from 
USIU tc rsu. Whether or not this w^as the most critical training 



ERIC 



5 



need or not was never established. Directors seemed to have 
a long and varied list of other training needs that they would 
have preferred to address, _ 

Daring each of the five meetings sponsored by LTl there was an 
attempt to elicit needs in a more or less informal fashion. 
Most of those needs were iirjr.ediate, institute-related concerns 
not the longer range needs and priorities of the profession (as 
the goal statement indicates) . 

During the site visits additional needs were identified but, 
again, these tended to be immediate institute-related concerns 
and not those of wider perspective. 

The opportunity for achieving this goal might have come out of 
th': October "training guidelines" meeting but, apparently, the 
product was a series of recommendations for the 1973-74 insti- 
tute/f ellowship/traineeship program. The guidelines are rnore 
procedural than prograjr^atio and therefore do not help to 
achieve this goal. 

Since no list of training needs and priorities exists, it m.ust 
b^ concluded that this goal was not achieved, Hov;ever, there 
appears to he coefficient outcroppings cuid indicators from tho 
various LTI activities to generate such a list. 

^ • coordinate a ll of the_ LTI ^'^pt jvities_ to provide of feci: ivo 

interch.^ng e of information b e t y;o e 1 1 t h e Bu r e au / the In s t i t u t e / 
a variety of constituencies, an d rolatod major professional 
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b odies and to provide these coniponents of the total trainincj 
ventvir e with appropri ate bull et ins and report s . 

Evaluation . This goal relates to the conimunication and clearing- 
hcuso functions. The primary relationships are: 

LTI to Institute Directors and Institute Directors to LTI 

BLLR to LTI and LTI to BLLR 
Secondary relationships are: 

LTI to ALA, AECT (and perhaps ASIS) 

LTI to other nontraining programs sponsored by BLLR and USOE 

While this goal was achieved, it was uneven in quality. Highest 
marks go to the LTI staff for being responsive to individual 
specific requests by institute directors. About half the direc- 
tors sensed that LTI could be a helpful resource for answers to 
questions about institute opurations, resource people, placement 
of graduates and, on occasion, interpretations of BLLR coTrununica- 
tions and expectations* Since responses to these special requests 
v;e>:e so timely and useful, the same individuals tended to come 
back for help. They v;erc satisfied v;ith the assistance they 
received. On the other hand, about half the directors did not 
know (or care?) about help available from the LTI office. 

The personal service seemed to stem from contacts made at the 
variovis meetings sponsored by LTI and from, site visits. The 
site visits were cited as being of special help in establishing 
liaison with LTI. 

The attempt to publish a newsletter met with mixed feelings. 
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Only throo specific "nemos" to cUroctors were identified. 
Several directors said thoy knew nothing about LTI until the 
notice of a tneeting arrived. Several said they had alrr.ost 
no contact with LTI except for the rnecitincj. The general 
an'iivalence regarding printed communications is best demon- 
i";trated by two directors: the first said the newsletter 
v;as useful and the second said it would be good to have a 
newsletter. The mode response was that the number of con- 
tacts with LTI were "about right" and the quality of the 
contacts was "helpful". 

CommunicatJ.on bt'ween LTI and BLLR is another matter. It 
appear.-? as if a variety of approaches were tried to optimize 
communication. These efforts can best be described as suc- 
cessful at times and unsuccessful at other times. The 
unevenness of the comm.unicat ions was due to the search for 
the best way to achieve it. Personality variables alv;ays 
affect communication and there is no exception in this case. 
No good would be accomplished to single out individuals and 
the barriers which each one used to screen messages. While 
the conflicts v;erc not major, th.ey pose warnings for future 
relationships. For instance/ BLLR feels, that I^TI's reporting 
Was inadequate during the first year while LTI feels that 
RLLK's paternalistic attitude caused blocks to communication; 
BLLK indicated tliat mcotings between tho two staffs v,'crc in-* 
frcquont, yet LTI v;as asked to stop attending BLLR staff 
m.cetings after being invited ^nd actually attending several 
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sessions. ThesG are typical percjptionn of ambiguous rela- 
tionships which wore well on their way to resolution at the 
end of the fiscal year. 

There is no evidence of LTI's influence on or cornmunication 
with professional associations other than appearance at ALA 
and several state library meetings but without substantive 
inputs. Individual staff people v;ere visible and conducted 
a considerable abount of business with institute represen- 
tatives but almost no interchange occurred between LTI and 
the associations. The relationship vH.th the Association 
for Educational Communications and Technology (AECT) was 
contractual in nature; i*e», the production of a sound 
filrr.strip. Even this relatively minor involvement between 
the two groups produced a positive affect both ways and is 
to be coiftraended as a first step. 

One fortuitous relationship w^\ch helped to achieve the 
goal was the use of library schools in Atlanta and Denver 
to serve as hosts for the meetings held there. This 
arrangement offered an excellent vehicle for interpreting 
LTI's program to new audiences. 

The goa] v;as achieved through members of th^i LTI staff 
interacting v:ith other programs. For example, the exchange 
of quarterly reports between LTI and the Center for the 
Study of Information and ^ ation (CST.E) at Syracuse 
University and the LTI Director's attendance at the National 
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Advisory J^oard meetings of CSIE served as an information 
oxchcincjG mechanism* 

To provide tochnical assist ance to other oncfoing £\LET (BLLR)^ 
training pro grams through appropriate activit ies such as 
consultations r on-s ite visits^ etc. 

E valuation : This goal V7as achieved more than any other goal. 
The LTI was most responsive to several special request ini- 
tiated by DLLR during the course of the year. These activi-- 
ties tended to extend the visibility of LTI and to fulfil 
DIjI.R expectations. The staff is to be commended for imme- 
diate response to technical service requests* 

One of the most significant successes in this area was the 
site visit program* All institutes wore visited and compre- 
hensive reports were written. This procedure allowed LTI 
staff to be briefed on progress in each of the institutes 
and helped institute staffs and participants to better 
understand the role and potential resources of the LTI. 
60^ of the Directors specifically mentioned the site visits 
as being especially helpful. No one indicated dissatisfac- 
tion* 

The meetings v/hich developed guidelines for paraprof cssional 
training and ins titutc/fellov;ship/ trainee ship guidelines 
vrere especially productive , The latter session produced 
"substantial inputs" for the 1973-74 guidelines. This ac- 
tivity must be designated as a success. The utility of the 
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paraprofossional guidolines is more doubtful* There is no 
evidencG o£ theii: use but this does not deny tha quality of 
the product, _ 

The assistance which LTI provided in organizing REFORMAi a 
group of Spanish--speaking librarians helped to create a quasi- 
official group which met for an initial meeting sponsored by 
LTI and a follow-vip meeting at ALA v/here future plans for the 
group were developed. While this effort required minimal 
contributions from LTI^ it can be viewed as a successful com- 
ponent of technical services ► 

The publication of Using Your N ation* s Capital - An Indexed 
Guide to MultiTT.edia Resources i n Vv'ashing ton, D. . was received 
by library science students and faculty in th-^ Washington area 
with enthusi£ism. This project was picked up from the previous 
year's contract and completed. The publication is well- 
organized and offers additional evidence of LTl's ability to 
respond favorably to technical service requests*. 

To res pond to sudden emerging training ne eds noted by the 
Bureau and/or components of the LTI; with appropriat e task 
for ce activit i es capab le of addre ssing such is su^^s • 

Evaluation ; This goal is actually very close to the previous 
goal statcinent and therefore / the above con^^.ents iipply. 

Of particular note v;as the "sudden emerging training need'' 
which brought together the r<j:FORi'L\ group and the ncods which 
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v;erc cUscovered daring site visits tincl handled imediatcly by 
LTI staff vhoAi its representative returned. The continued 
telephone calls and correspondonco from the directors were 
hatidled expeditiously and to the satisfaction of the requestor. 

To prov^ ide a variety of speci fic functions which will he lp 
imulctaent and broaden the effectiveness; of the IncStitute: 

—j^.^i... » II. « ■ . I ^ .mm . ■ _ ~ , » . II I . ■ I - ■ I --- . - I. ■■ . I I « I I I I 

e>q.^ the devel o pment and operation of relevant training 
activities? subcontractual activities as needed; the develop- 
ment of appropriate training models; an d the design and 
im plementation of leadership training act i vities focused on 
special priority topics . 

Evaluation: This goal appears to be yet another extension 
of the ** technical assistance" goal. Some of the "activitie.' " 
stated in the original proposal were subsumed under other 
goals, e.g., "preparation and dissemination of periodic news- 
letter," "revision of program guidelines," and "coordination 
with other Bureau activities to achieve national priorities 
related to minority library program development." These ele-- 
ments have already been assessed. 

"The preparation and dissemination of selected training pro- 
gram analyses" was defined as the organisation and editing of 
previous Title ITb reports {from ERIC) to provide guidance to 
present and future directors. This has been referred to as 
the "what we have learned" project. This project was begun 
during the current fiscal year but wa ; not finished and there- 
fore it is not possible to judge its contr.lVjution* 
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The *'dovelox:)ment of institute training models" was accomplished 
in tho 'preparation and dissenination of selected training pro- 
gram analyses*' and incorporated into the 1973-74 guidelines 
v;hich is an indicator of success. 

The u^ajor acco^tiplishment in this goal area is the ''development 
of appropriate leadcrfihip training materials*' and the "corrjnis* 
sioning of subcontractual activities." The v;riting, testing 
and revision of an evaluation manual was one specific product 
of this effort. Also, the creation of a communications train- 
ing package (COMPAC) was an additional product, (Evaluation 
of the products is located in a separate section of this report*) 
The comi\mnications training package was subcontracted to Concern, 
Inc. The production of a sound filmstrip regarding the Phila- 
delphia Action Library, Building Bridges to the Future ; was 
subcontracted to AECT, 

The evaluation on this goal must rest on tho "development of 
appropriate leadership training materials." Since subcontract 
rctivitics were carried oat, one might say that this goal was 
reached. The quality of tho production and the tim.eliness of 
the delivery is discussed elsewhere. 

The comments of the directors would give rise to questioning 
the appropriater.oss of the materials developed. The need for 
the materials v:as based largely on the perceptions of the LTI 
staff, some of whom had heard expressions of need in these 
areas during the previous year. Some directors felt that 
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"evaluation v/as the answer" no Matter what other needs were 
articulated. On the other hand; the evaluator explored other 
ncedii whilo intorviowincj the directors and received a laundry 
list of some twenty discrete iteir.s. Perhaps evaluation and 
coirjrtunication skills are the most appropriate areas in which 
to develop training materials. There was sufficient expres- 
sion of anticipation from the directors that would cause one 
to believe that these materials are welcome and will be used. 

The basis for the production of the sound filmstrip on the 
Philadelphia Action Library was never explained by anyone 
interviewed except by the nebulous phrase that a former 
Bureau employee wanted to have the project documented for 
dissemination purposes. 
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IMPACT EVALUATI0>3 



Objectives (or goals) con be stated and achieved but unless 
those gov^ls are significant and contribute to the common welfare of those 
involv^ed^ achievement is a hollow exercise. The goals of the LTI for 
1972-73 were important as well as flexible. The flexibility penaitted 
the Bureau and LTI staff to move as emergent needs dictated. From the 
analysis and evaluation in the previous section it can be said that the 
goals were achieved to a high degree and that if the label of success 
can be assigned to attainment of goals, LTI was successful. But there 
is more. 

\<fhat difference has LTI made in the library world? Are people 
{involved with institutes) any better because LTI existed? If LTI 
v;eren't therc/ would it be missed? Is the investment paying off? 
Questions like these get to the heart of an evaluation - the impact on 
those who cone into contact with LTI staff and activities during the 
ye:ir» 

V-^hen the LTI Director was asked about his disappointments during 
the past year, he responded by saying ^ ''There *s so little evidence of 
success." This is partly true (and will always be so) in any educational 
enterprise which involves interaction betv^een teachers and learners. The 
results of the rolationsViip cannot be observed for many years. But there 
are evidences of success v:hich crop up as indicators during interviews 
and conversations. The inpact evaluation is based on these indicators. 
It must of necessity be more subjective since data are fraomcntary and 
inferential. It m^ay be more importaiit than the discrepancy evaluation 
which is much m.ore objective and data-based. 
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On the Concept of Xmuact and LTI 

Very rarely can there be impact without i^oina action. Using this 
truis:m ay a basis one miyht ask, "VJhat did LTI do and \v'hat effect did it 
have as a result of these actions?" An analysis of LTI's list of acti- 
vities v?ould yield sorae irrjne^iate short tern effects (which were largely 
discussed in the discrepency evaluation section) and soir.e long term 
effects (which have yet to be assessed). 

This was a year of role exploration for LTI. V/hile the goals stated 
in the original proposal provided some direction, they permitted a wide 
flexibility in program development, V^hile perception of LTI varied among 
all v;ho came in contact with the staff, there was a strong conceptual 
unity tliat LTI is a consultative and information clearinghouse organiza- 
tion, closely related to the Bureau, \^ho exists to help improve library 
training programs. The perceptions were developed by communication with 
and observations by individuals outside LTI. It is interesting to note 
hov; closely the perceptions of reality and the intentions of the proposal 
arc matched. 

This year was spent largely in search of an appropriate role for 
LTI. Changes of personnel at the Eureau, transfer of LTI from USIU to 
Florida State University, new staff for LTI, and a new batch of directors 
caused some confusion as to what LTI was and was going to bo doing. That 
there is exrv/^ agreement regarding role among the various constituencio."j at 
tliiri point in tim.c is a credit to the jrersistcnco, visibility and consln- 
toncy of LTI personnel. 
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Impact o n Directors 

At the vory least the directors oC library institutes arc aware of 
I/fl and know that they sponsor meetings which are primarily concerned 
with evaluation. At best a ir^ajority of the directors are satisfied v;ith 
the services they have received as responses to specific requests. Be- 
tween the two extreines about one-third of the directors know very little 
about LTI, are confused about its role or view it as unrelated to their 
personal institute concerns. The evaluator sensed a feeling of growing 
understanding and acceptance of the LTI on the part of the directors. 
Expressions of anticipation for forthcoming produces were evident. The 
directors are beginning to look for more leadership from LTI and expect 
to find it during subsequent years. 

Impact on the Bureau 

There is one concept v;hich seems to permeate the thinking of Bureau 
personnel in regard to the LTI - confidence in_ the staff , \Vhile there 
have been occasional feelings of miscommunication or poor corrot^unication 
and minor frustrations regarding the handling of several activities, the 
BLLR staff has demonstrated tolerance for the problems because they were 
dealing v;ith reasonable people. During the metamorphasis of the 1972'-73 
LTI, the Bureau viev;od this group as a resource for institute directors 
and as an "extension" of itself in the sense that LTI could perform some 
of the professioral activities which the Training Division of BLLR vrould 
like to have perform.ed bit were unable to do becaut^e of staff and fiscal 
constraints* The requests for technical services which emerged during 
the year demonstrated confidence in LTI or else they would not have been 
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approached to handle such delicate problems as the organization of 
Spanish-ypenkinq librarians. Tlie Bureau perceivos LTI as "an emerging 
unit" v;hioh servos as an infornatioii clearinghouse on library training 
and as a vehicle for reaching institute directors. Ideally, training 
needs of the profession will be defined through LTI and priorities will 
be developed by the Bureau. This has not yet occurred. 

Impact on the Profession 

This is one of the long tern effects. The ultimate iirpact will 
come as institute participants begin to make professional contributions 
resulting from experiences during the institute. If any credit can be 
assigned to LTI it will be remote and obscure. V?ho knows what iippact a 
Compac film or a technique learned at a workshop might »have had on one 
participant? This is difficult to judge. 

The shorter term effects are easier to assess but much less drama- 
tic. In this evalviation no systematic attempt was made to assess the 
impact on the profession. Casual observation and conversation v;ould 
lead to the opinion that there has been very little impact on the pro- 
fession as a whole as a direct result of LTI's efforts. 

Impact on the LTI Staff 

Impact of LTI on LTI staff? Yes, this is a vital component of the 
evaluation. LTI is an all- :onsuming activity for the ir.taff. The dedi- 
cation and ;iost dor.ionst rated by each person is a dcligrit to observe. 
During the interviev;s each person mentioned working u'ith others on the 
staff as one of the satisfactions. There is a good espri t de corps and 
mutual respect for one another. LTI has become mission for each 
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individual involved. The basic concorns are people concerns, products 
seem to bo secondary • Each person has unic|ue professional capabilities 
and unique perfionality charactcrij^tics and all these qualities appear 
to be coinplcraentary. LTX lias become a way of life for each staff member 
This sense of mission, professional capability, and warm, human concern 
is conununicated to those who associate with the staff. The result is 
the establishment of empathetic relationships which assist in accomp- 
lishing LTI goals and increasing its impact on others. 
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PRODUCT EVALUATION 



Four major products v?ore initiated/ dovoloped and produced during 
the 1972-73 fiscal year: 

1. Planning and Evaluating Library Training Programs ^ an evaluation 
nanual for institute directors 
Con^pac ~ a coimunication skills package 

3 . Using Your Nation's Capitol; An Indexed Guide to Multimedia 
R esources in Washington, D. C> 

4. A sound filmstrip, Building Bridges to the Future 

Planning and Evalu a ting Library Training Programs ^ 

The manual offers a corT?prehensxve overview of the evaluation process 
with special attention to library training prograirs, \Vhile Stuff lebeain' s 
CIPP model (Context, Input, Process ^ Product) is emphasised, the contr?. ■ 
butions of other evaluation specialists are appx'opr lately noted. Most 
evaluation processes are theoretical until they are app3ied. This manual 
is no exception* In order to overcome the theoretical stigma, extensive 
examples are used. The examples are based on library training programs. 

Special corrjT^.ondation should be made for the way in v;hich the raanual 
was developed. All persons v;ho participated in th : process are credited. 
The message is that this volume is not the product of one person's think-- 
incj, one porL^on's prccodure^i, a^id one periion's values. The diverse con- 
cernij of the group for which it intended were factored into the synthesis 
of the process^ evaluation approach* 



This assessrr.ont was ir.ade using the "working copy" of the manual^ not 
the f.inished product* 
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Tho volume is a happy compromise between providincj scanty superfi- 
cial inforrration and overly comprehensive information v;hich tells the 
reader more than ho/slie wantc to know. This is a practical handbook for 
directors of library institutes - nothincj more, nothing less. 

The generous appendices should also be useful for the directors. 
Again, specific examples offer touchstones for evaluation formats which 
might be developed for tho unique requirements of each institute. 

The real test for this product v;ill come with its use* The draft 
appears to be one of the best publications of its type, 

Compac 

This multimedia package was developed in response to a need per- 
ceived by the LTI staff. It is designed to be used in library training 
programs over approximately three days for the purpose of increasing 
awareness of the need for better communications both within the profes- 
sional education program itself and as individuals deal with clients in 
library environments. It contains a 17 minute film (TRANS action) , a 
slide/tape presentation, a videotape, audiocassettes and a teacher- 
participant manual. 

Unfortunately, the entire package has not been coripleted and there- 
fore could not be evaluated. Only tho film was available for preview. 

To preview and evaluate a film out of context is a dangerous pro- 
cedure,- especially when it is part of a larger system. This film almost 
stands alone as an introduction to transactional analysis. It draws 
heavily on the publication, Co.TLmu n i cat ion , an issue of the Kaiser Al umi- 
num News edited by Don Pabun. It is a useful film to introduce basic 
concepts in comiriunications, with emphasis on transactional analysis. 
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Tho library applications are obvious end appropriate. There are some 
tecluucal shortcovaings which are supposed to be changed before final 
releayo , 

The LTI staff is proud of their efforts in developincj thi.s package 
which has been subcontracted to Concern, The directors look forward to 
using tho materials. 

Using Your Nation* s Capitol; An Indexed Guide to 
Multime d ia Resources in Washington^ D. C> 

Tb.ere is probably no reference work like this resource. The many 
and diverse information resources of the Washingtoni D. C* area are well- 
organised and described thoroughly. There are more sources than this 
evaluator has ever seen in one publication before. The specificity of 
the information is a major asset: naxnesi addresses, telephone numbers, 
functions/ services and titles of additional references in all media 
formats. 

The obvious limitation, listing sources only in the Capitol area, 
may not be as sever a deterrent to using the reference as a listing of 
sources in L^avenport, Iowa, Since the Nation's Capitol is a resource 
for the entiro Nation, individuals from all parts of the country should 
have access to V7ashington' s rcr:ources. Many people from all states do, 
in fact, write and visit many of the organizations listed for assistance. 
But there is a definite advantage for people living and working in the 
Washington, D. C. area. 

The test of this voli\me's utility will be in the actual use it 
receives by those who road it. it docs have the potential for being an 
extremely helpful volume, to those v;ho are trying to find specific sources 
in the W-ashington area. 
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Buildi ng bridges to the Future 

This sound Ciln^strip, produced by AECT on a subcontract from LTI, i« 
a good visual report of the Philadelphia Action Library^ It is designed 
for librarians and interested individuals in any community. Librarians 
vho view the filmstrip should be moved to consider new services which 
they might offer to the community^ For the community, it should help 
to develop interest in and support for a new type of library with ex-- 
tended services. 

The filmstrip is technically excellent. The visuals are appropriate 
and the audio is good. The use of actual voices of the people involved 
adds credibility to the presentation. 

Plans for distribution of the filmstrip were not available at the 
time of evaluation. 



C0>3CLUSI0NS AND Rt:COM!'lENDAi^ION-S 



Conclusi ons 

!• Most of the 1972-73 goals for LTI were achieved to a fairly high 
degree. 

2. The goals were sufficiently flexible to perniit additional acti- 
vities to be performed as needs emerged. 

3. The LTI did create in the directors a new awareness of evalua- 
tion. 

4. Training needs and priorities were not determined in direct 
response to 1972-73 director's needs/ 

5v LTI provided a useful, personalized information service for 

directors v?ho mads specific requests. 
6. LTI provided substantial input for BLLR training guidelines. 
7* CoirjT^unication between LTI and the directors was adequate? 

between LTI and BLLR it was uneven but there was evidence of 

improvement toward the end of the year. 

8. LTI was able to respond quickly and adequately to BLLU's 
requests. 

9. LTI did not derronstrate any visible impact on the library 
profession. 

10. The products generated during the year v;ere well done but not 
all were completed. 

11 . The LTI staff was competent, helpfi:!/ cordial, and skillful 
in handling its varied activities. 
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ft ConGlusio n to the Conclusions 

In the final analysis one inuc^t raisG tha question, ''Was LTI worth 
the invostrp.ent?" The evaluator must answer with some dcgrue of subject- 
ivity. "Yes, there was responsible stewardship of the funds and along 
with many spocific accomplishments, there were contributions which niay 
never be measured." 

The trauma of transition from one institution to another? the ambi- 
guity of LTI's role; the clarification of communication channels among 
diverse groups in diverse parts of the nation; the logistics of staffing 
and location - all these events and problems were overcome and still 
there was a good yield of LTI visibility, direct assistance to directors/ 
direct help to PLLR in response to immediate needs, several product :i and 
a viable organization to better meet the needs of the subsequent year. 
This year must be considered as a year of transition/ clarification and 
new direction. LTI has an excellent, cohesive staff v;ho share common 
goals and respect the abilities of one another. The proposal for the 
second year reflected lessons learned during year one and developed more 
specific objectives based on experience. The Bureau also appears to be 
more satisfied now that tho LTI is established and moving in consort with 
its program. 

After assessing goals/ impact and products, there ia a feeling that 
the v;hoip is grealror than the sum of its parts. 

Re coniraendat ion s 

Several of the recommendations have already been considered and in^ 
corporated into the 1973-74 design for the LTI. They are included in the 



total list as a matter of record. 

1. Establish nore realistic, less global goals. Stateraents of 
objectives v;oi!lcl offer bette guidance and better serve the 
purpose of evaluation. 

2. Evaluation guidance should } ^ provided through the evaluation 
ir.anual and on an individua-^ basis as requested. Do not 
repeat the workshops on the evaluation process. 

3. Deliberately move to determine need?, and priorities in the 
library field, (A rr.eeting of a5;sociation executives and/or 
deans of library schools V7ould help to move toward this 
objective.) 

4. Continue technical services to directors and the Bureau; 
retain some flexibility to respond quickly to emergent needs* 

5. Continue the site visit program, 

6. Develop a regular newsletter to maintain contact with direc- 
tors and to serve an exchange of ideas among institutes. 

?• Field test the products developed this year to determine the 
degree to which they achieve the original objectives and to 
make recomiT.endations for optimum use based on the field test 
data. 

8, Through intimate contacts w»ith the library field LTI is in a 
position to help the Bureau determine its role in training, 
A plan should be mounted to achieve this end, 

9. Plans should be instituted to develop a stronger im.pact to 

the library profession within the framework of the LTI program* 
10* Consider ways in v^hich LTI might be able to assist directors 
in handling the cross-cultural situations which characterize 
most institutes* 



Continue relationships with institute directors which began 
1972-73 oven though some institutes have terminated. 
Help institute directors to document activities for purposes 
of reporting and sharing ideas. 

Initiate activities based on observed needs, not just those 
which the Bureau perceives as necessary. 
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Evaluation Plan 1972-73 

Letter to institute Directors 

Persons Interviewed 

Interview Schedule for Directors 

Interview Schedule for BLLR Staff 

Interview Schedule for LTI Staff 



K VALUATION PrJ\N 
LTI 
1972-73 



Since the evaluation must necesyarily ba Pi>st hoc^ two iRv^jor diraen- 
sions v;ill be assessed: 

1, Discrepancy (thf» degree to v;hich original objectives 
o£ LTI were achieved) ; and 

2, Impact (the inwiediate effect of the LTt on its 
primary and secondary audiences) 

Discrepancy Evaluation 

Each cjoal and its related activities will be examined to determine 

whether or not it has been accomplished and if it has/ the degree of 

success. The procedures for gathering this information will include: 

1. Personal interviews with each staff member and BLLR staff 
concerned with training 

2. Telephone interviews with each of the institute directors 

3. Questionnaires from each of the regional institute participants 
The line of questioning for each of the procedures will be derived from 
tVie goal and activity statements as they appear in the proposal. For 
example/ the first goal is concerned v/ith "more adequate leadership 
training skills and techniques for institute directors..." and the first 
activity has to do with "identification and assessment of training needs 
Questions night be: 

- ITnat specific training needs have* been fulfilled by LTX? 

- Are there other training needs which ought to receive high 
priority ati the LTI develops its plans for this year? 

- At the beginning of your institute/ in which of the following 
areas of training did^ou need help? (Rank order) ovalua- 
tion c orrjnunication skills use of audiovisual media 

i nterpersonal relations ^ ^institute management 

Innovative practices o thers 



KVALUATION PLAN - LTI - 1972-73 Page 2 

Rank order the list as you perceive your training needs today. 
Product i:valuatIon 

Kach of thci products will be reviewed along with field test data. 
The evaluation n^anual will be reviewed by two evaluation specialists and 
the I.TI evaluator. The conununication skills package will be reviewed by 
the LTI evaluator and two specialists in communication skills. Since 
these products have not been used in their final foi.ms, evaluation will 
be performed on their technical quality and inpact potential . 
Impact E valuat \on 

The impact dimension will detenrtine intended and unintended outcomes 
of LTI activities on (1) institute directors; (2) institute participants; 
(3) faculty members not directly involved at institutions where institutes 
were held; (4) staff of the Bureau and (5) staff of LTI. 

Each group will be identified as completely as possible and a sample 
will be determined depending upon the total population. The initial 
sample v;ill be contacted by telephone while the remaining number will 
receive c|uostionnaircs. 

The litie of questioning will be open to permit maximum flexibility 
of response. For example, the evaluator will ask for specific examples 
of new or modified behaviors which can be directly attributed to LTI 
influence • 
Validation Gheck 

The data from the discrcpency and impact analyses will be synthe- 
sized and tentative evaluation statements will be made. Then, the data 
on file in the LTI office v/ill bo examined to determine the extent of 
congruency between the two data sources. 
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July ySf 1973 



Vrom: 



1 liave bc^.MX ask::d by Harold Goldviue5.n to the *'outside" ovaluator 
' or th-i i.;iad:^r3'a.lp Train Ln';^* :Cuotit:\ite (LTD fur^ded by the. Bureau of 
.r,j.bra^:lo3 arid Loar^ning RosoMrct^s of USOf:\ The nurrx^iUi or this evalu- 
ation is to roport back to Lh„x l-TX tihaff how well i:hey performed their 
rosponribili t ieu this p^^cst yaor. If the uvaluabion is dorrs well/ th*:! 
raedbac'C s^iouiJ heip to improve next year* 3 activxtios, 

I \;xii be ciliinq you :*^Oi^eti.a^i wi<:hin the next two weeks to ai;k you a 
row cU'i:^tLo:>s rt^gardia^: :.ha a<;3i3t-aac£* v^hich LTI has provided during 
thr: pi\>s^: ^"a j'^on^hy. Your r^^iponr^^^s r.hould ba candid and consttuctive, 
Vv:;u wil l not ba quoted with idan ti t: ication unle^^ you rcqaa^t it. 

Tlie call ohould taky lS-'20 rainutey* If it ir^ not convenient to tai.k 
u-hcin 1 ca]..\, parhaps v;a could arrange for a Gpecific tirre later that 
day or th^ *\"}<t day* 

^haiik you in advanca [or your contribution. I iim convinced that our 
rrcop'^rative offot'ts can yield useful quidaaco for our LTI collaayue^'. 

sinoer-aly, 



Building Skylop | Syracuse University ] Syracuse, NewYork13210 | (315) 478-6524 - 



Persons Interviewed 



Directors 



Elise Barrette, East Tennessee State University 

Alvin J. Goldwyn, Case-Western Reserve University 

Norman Higgins, Arizona State University 

Virginia Lacy Jones, Atlanta University 

John R. Loughlin, University of New ilampshire - 
Merrimack Valley 

Junius Morris, Highline Community College 

Patrick S. Sanchez, California State University - 
Fullerton 

Laurence Sherrill/ University of Wisconsin - 
Milwaukee 

Fleming A. Thomas, Burlington County College 
Tommie Young, North Carolina Central University 



8/10/73 
9/04/73 
8/03/73 
9/04/73 

8/03/73 
8/03/73 

8/09/73 

7/30/73 
8/16/73 
8/03/73 



LTI Staff 



Dorothy Anderson, Associate Director for Coordination 
Bone Durant, Field Consultant 
Harold Goldstoin, Director 

Brooke Sheldon, Associate Director for Training 



9/24/73 
7/17/73 
8/02/73 
8/02/73 



BLLR Str.ff 



Dick Hayos/ Acting Associate Commissioner 

Frank Stevens, Program Manager, Library Training 



8/01/73 
8/01/73 



Interview Schedule for Directors 



I'm Don Kly, the "outside*' evaluator for the LTI, The purpose of 
the evaluation is to report back to the LTI staff/ how well they per^ 
formed their responsibilities this past year. If the evaluation is done 
well; the feedback should help to improve next yearns activities. The 
evaluation is not for USOE/BLLU directly, although they will see a copy 
eventually. I am trying to help the LTI staff to know what worked; what 
didn't work; and the degree of success or failure. Your narae will not 
be identified separately unless you request that I do so. 

!• Think back to the time when you learned that you would be running 
an institute, ^^hat kinds of needs did you have at that point? 

(Let rae illustrate further - information on evaluation? ; ways 

of teaching corrj:;unication skills? ; interpersonal relations? 

; use of audiovisual media? } institute management? ; 

recruitment? ) What were the two most important needs? 

2, What specific training needs have been fulfilled (partially or 

completely) as a result of LTI*s efforts this year? (Are you doing 
anything differently because of LTI?) 

3* Are there areas which LTI has not considered which ought to be? 

4* Can you describe for me the priorities for critical training needs 
for the library and media professions? 

Is this your list or does it reflect the thinking of the BLLR? Did 
you participate in any formal discussion regarding these priorities 
sponsored by LTI? 



Directors . p^^^^ 2 

5, Approxiiaately how many comunications did you recoivo from LTI this 

past yec\r? nev;slettors and general mailings ; personal letters 

} site visits ? 



Could you describe in your own words, what LTI is and what it does? 

Do you wish you had more frequent contact with LTI? (less frequent?) 
V/hat could LTI provide you that no other organization could? 

6. What specific requests have you made of LTI this year? What response 
did you receive? Are you aware of the various programs in which LTI 
is involved beyond the ones you mentioned above? (Yes) What are 
.they? (No) Let ne tell you about . • • REFORI-IA, evaluation, a 
communication skills kit, etc., etc. 



Interview Schedule for BLLR Staff 



1. Here are the initial goals for LTI during the past fiscal year. 

- Was the goal achieved and to what degree? 

- V/as the goal ir.odified? 

- V'/ill (should) the goal continue through next fiscal year? 

2, Specifically (if not covered above) 

- To what extent did LTI contribute to the establishment of 
BLLR training priorities? 

- To what extent did LTI recommendations for training guide- 
lines emerge in the final guidelines? 

- What technical assistance was particularly helpful? 



• 
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Interview Schedul;ii for LTI Staff 



1. Do you have any reactions or suggestions regarding the evaluation 
procedure outlines in my statement? 

2. V?hat has your primary role been in LTI this year? What did you 
actually do? 

3. What are some of the satisfactions which you derived from your work? 

4. What were soiae of tlie frustrations and/or disappointments? 

5. What and how were needs identified and how were they incorporated 
into this year's prograin? next year's proposal? 

6. What will you do differently next year as a basis of this year's 
experience? 

7. What are some of the outstanding programs when taking everything 
into account? Which one(s) stand out in your mind? 

8. What "payoff do you feel will stem from these programs? (Are they 
worth the government's investment?) 



